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ABSTRACT 

This review examines 12 programs designed for use in 
classrooms to develop "general thinking skills" used in learning and 
problem-solving situations inside and outside classrooms. Analysis of 
program features focuses on the extent to which each program shows 
promise in helping students transfer the skills they develop through 
completing program exercises to other learning situations. It is 
argued that programs with multiple goals are more likely to promote 
transfer of skills than those that promote the single goal of 
particular skill development. Thus, programs that also focus on three 
additional goals (developing metacognitive awareness and strategies; 
conditional knowledge of contexts for knowledge and skill use; and 
attitudes and dispositions conducive to using the skills) show more 
promise. Other program features that increase the likelihood of 
transfer are (1) program materials are content related; (2) teaching 
methods include a high degree of teacher mediation; (3) transfer of 
skills is directly taught; and (4) teachers are trained to use the 
materials. It is concluded that interventions to teach thinking 
skills can be useful tools, but they need to be used in conjunction 
with occasions for students to develop and use higher order thinking 
in subject area learning. (Forty-eight references are attached.) 
(Author) 
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Abstract 

This review examines 12 programs designed for use in classrooms to develop 
"general thinking skills" used in learning and problem-solving situations in- 
side and outside classrooms. Analysis of program features focuses on the ex- 
tent to which each program shows promise in helping students transfer the 
skills they develop through completing program exercises to other learning 
situations. It is argued that programs with multiple goals are more likely to 
promote transirer of skills than those that promote the single goal of par- 
ticular skill development. Thus, programs that also focus on three additional 
goals (developing metacognitive awareness and strategies; conditional knowledge 
of contexts for knowledge and skill use; and attitudes and dispositions condu- 
cive to using the skills) show more promise. Other program features that in- 
crease the likelihood of transfer are (a) program materials are content-re- 
lated; (b) teaching methods include a high degree of teacher mediation; 
(c) transfer of skills is directly taught; and (d) teachers are trained to use 
the materials. It is concluded that interventions to teach thinking skills can 
be useful tools, but they need to be used in conjunction with occasions for 
students to develop and use higher order thinking in subject area learning. 
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INTERVENTIONS TO TEACH THINKING SKILLS: 
INVESTIGATING THE QUESTION OF TRANSFER 

Cheryl Rosaen^ 

The Question of Transfer 

In recent years a wide variety of instructional programs have been devel- 
oped specifically for the purpose of improving students' learning and thinking 
skills. Many of these programs emphasize developing students' ability to focus 
on and effectively use thinking processes, with less emphasis on the content or 
problem being solved. While there is some evidence of internal success in pro- 
grams that develop general thinking skills and strategies, further investiga- 
tion is needed to determine whether isolated skill development helps students 
to apply abilities and strategies beyond the program exercises; that is, to 
what extent do students develop the disposition and capabilities to use the 
skills they develop through program exercises in other learning situations or 
to solve everyday problems? 

The research agenda in the Center for the Uarning and Teaching of Elemen- 
tary Subjects focuses on studying ways to improve higher order thinking and 
problem solving in five subject matter areas: social studies, mathematics, lit- 
erature, science, and the arts. As part of the Center's broad efforts to exam- 
ine the nature of instruction that promotes higher order thinking and problem 
solving in the subject matter areas, this paper discusses the role thinking 
skills programs might play in improving learning and thinking in the 



Cheryl Rcsaen is a senior researcher with the Center for the Learning 
and Teaching of Elementary Subjects and is assistant professor of teacher educa- 
tion at Michigan State University. The author acknowledges the assistance of 
Jere Brophy and Ralph Putnam for comments on a draft of this review, and of 
Danise Cantlon for comments on drafts and valuable bibliographical assistance. 
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subject areas. Specifically, this review examines 12 programs designed for use 
in elementary classrooms to develop "general thinking skills" used in learning 
and problem solving situations both inside and outside classrooms. Analysis of 
the programs focuses on the extent to which each program shows promise in help- 
ing students transfer the skills they develop through completing program exer- 
cises to other learning situations. 

The ability to transfer general skills and strategies requires having the 
capacity and disposition to use and apply them to learn and think in the con- 
tent areas, and in this sense is the ability to use "higher order" thinking 
(Resnick, 1987). An underlying assumption is that students must develop intel- 
lectual resources (e.g., thinking skills and strategies), and the resources 
must be accessible so that they can draw on and use them in learning situations 
(Prawat, 1988). Moreover, in a comprehansive review of the literature that re- 
lates to the interaction between domain-specific and strategic knowledge. 
Alexander and Judy (in press) identify and explain several ways in which 
learners' developing domain and strategic knowledge shape each other. Among 
several hypotheses developed in the review, they make the case that perceiving 
the relatedness in domain and strategic knowledge across tasks and across 
domains seems to characterize competent performance. Thus, they argue, 
acquiring knowledge seems to enable learners to abstract or represent mentally 
a given problem, to classify problems on the basis of their underlying 
structures, and to use the perceived relatedness to guide their performance. 
Given these apparent relationships among declarative, strategic, and 
conditional knowledge, it is important to consider how programs in "general 
thinking skills" contribute to subject matter learning. 

In this review it ia argued that a "promising" program is one that not 
only helps students develop specific thinking skills, but also helps them 
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deliberately generalize and apply the thinking principles they learn through 
program exercises to further their learning in the subject areas and to further 
their everyday reasoning abilities. Ii. addition to reviewing available evalua- 
tion data for each program regarding its effectiveness, particular attention is 
paid to identifying and describing program features that increase the likeli- 
hood that students will develop the disposition and skill to draw on the 
intellectual resources they develop. 

Program Selection and Analysis 
The 12 programs reviewed (shown in Table 1) were chosen because they are 
typical of efforts to improve general thinking skills and strategies, they have 
been widely used, and there is some information available about their effective- 
ness. In keeping with the Center's focus on elementary teaching and learning, 
this review is limited to those programs that are suited to students of elemen- 
tary age. 

Discussion of the 12 programs is divided into two major sections that re- 
flect differences in program goals. As shown in Table 1, the four programs 
listed at the top (Structure of the Intellect, Cognitive Research Trust. Think 
About, and Science ... A Process Approach) focus on a single goal : to de- 
velop particular thinking skills that will improve thinking ability (see Goal 
A). The seven programs listed at the bottom of Table 1 (Instrumental Enrich- 
ment, Productive Thinking Program, Odyssey, Talents Unlimited, Philosophy for 
Children, Reciprocal Teaching, Expository Writing Program, and LOGO) focus on 
multiple goals. In addition to trying to help students develop particular 
thinking skills (as programs in the first group strive toward), these programs 
take on additional goals that may include one or more of the following: develop- 
ing metacognitive awareness and sti.ategies (Goal B) , developing conditional 



Table 1: Overview of Program Goals 



Group I: 
Programs wH 
Single Goals 



Group II: 
Programs witl 
Multiple Goal 



Programs Reviewed 


Goal A: Devek)p 
Cognitive Abilities 


Goal B: Oevebp 
Metacognitive Aware- 
ness and Stratonioc; 


Goal C: Oevetop 
Conditional Knowledge 

KJI VAilllUAlO lui woo Ol 

Knowledge and Skills 


GoalD: Develop 
Attitudes and Disposi- 
tions to Use Skills 


oiiuuiuiu m ine inieiieci 
(SOI) 


X 








Cognitive Research 
Trust (CoRT) 


X 








ThinkAbout 


X 








Science...A Process 
Approach (SAPA) 


X 








iiibiiuiiiuiiidi cnricnnieni 
(IE) 


X 


M 




— 

(X) 


Productive Thinking 
Program 


X 


X 




(X) 


Odyssey 

Talents Unlimited (TU) 

Philosophy for Children 

Reciprocal'K "ohing 

Expository Writing 
Project (EWP) 


X 
X 
X 
X 

X 


M 
M 
M 

X 
X 


(X) 
(X) 
(X) 
(X) 


(X) 
(X) 

(X) 


LOGO 


X 


X 


rf 



Key: x Primary Goal of Program 
(x) Secondary Goal of Program 



, ^ knowledge about contexts in which knowledge and skills might be used (Goal C) , 
•and developing attitudes and dispositions to use the skills (Goal D) . 

For each program the following descriptive informa'-ion is given: the un- 
derlying assumptions and goals of Che program, the methods and materials used, 
the student population for which it is Intended, the qualifications teachers 
need to implement the program,' and evaluation efforts and information about its 
effectiveness. Throughout this discussion, it will be argued that programs 
with multiple goals (shown on the bottom half of Table 1) are more likely to 
promote transfer of skills than those that promote the single goal of particu- 
lar skill development. Moreover, illustrations are provided to show that pro- 
grams with the following characteristics are more likely to promote transfer or 
higher order thinking in the content areas and in everyday reasoning: (a) pro- 
gram materials are concent- related, (b) methods include a high degree of teach- 
er mediation, (c) transfer of skills is taught directly, and (d) teachers are 
explicitly trained to use the materials. These features are summarized for 
each program in Table 2 and discussed in detail throughout the review. 

In the concluding section of the review, features of more promising pro- 
grams are highlighted, with emphasis given to ways in which these features 

promote transfer. In addition, evaluation issues are identified and discussed. 

Programs Designe d to Improve Coenitive Abilities 
Four programs are reviewed in this section: The Structure of the Intel- 
lect Program (SOI), Cognitive Research Trust (CoRT) , ThinkAbout, and 
Science ... A Process Approach (SAPA) . These four programs share a common 
single focus on developing particular skills that will promote better thinking, 
with the overall goal of developing and improving the learner's capacity to 
think and learn. Program developers assume that improved capacity will allow 
students to engage in and monitor more complex cognitive activity as they 
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Tabla 2: Summary of Profiram Charactoristjcs that incraa$« Likalihood of Transfer of Skills to Other Contexts 
Progr^nsa^vie^ Proj^am Goals Materials Used 



Group I: 
Programs vwth 
Single Gocrfs 



Group II, 
Programs with 
Multiple Goals 



Stnicture of the Intelect 

(SOI) 



Cognitive Research Trusi 
(CoRT) 



ThfnkAbout 



Science..^ Process 
Approach (SAPA) 



Insirumeniai Enrichmem 
(IE) 



Productive Thinking 
Program 



Odyssey 



TalenU Unlimited (TU) 



Phitosophy for Children 



Radprocal Teaching 



Expository Writing 
Prpiect (EWP) 



LOGO 



A B CO 



Content Subject 
Fre9 Matter 
Related 



« («l 



(«l 



(X) 



X (X) (X) (X) 



X (X) (X) (X) 



X (X) (X) (X) 



X X (X) (X) 



X X (X) 



Intended Degree 

of Teacher Medtabon Transfer of Skills Teacher Training 

Assumed Taught 



Low Medium High 



Key: x Primary Goal o( Pro^^am 
(x) Secondary Goal of Program 



(+) 



Charactfiristics Show 
f^ogram Evaluation Increased 

Results Likelihood of Transfer 



Through Explicit 
Materials Training 



Required Gains Shown 



Some Transfer 
Immediate Shown 



(+) 



+ Program has characteristic Goal A: Oevetop Cojyiitive Abilities 
(+) Program shifting toward Goal B: Develop Metacogniiive Awareness & Straieoie 
devsrfopwg characteristic Goal C: Devetop Condilionai Knowledge 
GoalO: DevetopAttitudes and Dispositions 



(♦) (+) 



(+) 



1» % 



Shaded portions indicate characteristics inportant 
to transfer of skiUs to other coniQxls 
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encounter future learning experiences in or out of school. The particular 
skills on which each program focuses are summarized in Table 3 and elaborated 
in che sections below. 

The four programs are further grouped according to similarities in their 
methods and materials used. The first two (SOI and CoRT) are programs that de- 
velop abilities through content-free or generic materials. The focus of teach- 
er and student attention is limited to practice of particular skills with lit- 
tle emphasis on developing awareness of how and when to use particular strat- 
egies. The second two programs (ThinkAbout and SAPA) develop abilities through 
subject matter-related materials, but emphasize skill development over knowl- 
edge development. Moreover, these two programs have a higher degree of teacher 
mediation than the first two, where the teacher's role is to foster reflective 
thinking by providing opportunities for students to engage in reflective think- 
ing, providing appropriate guidance and support for developing effective think- 
ing strategies, and helping students become independent learners and thinkers. 
Thus, the second two programs contrast with the first two in the amount and na- 
ture of the discourse surrounding the exercises offered in the program. Con- 
trasts in program features, summarized in Table 2, will be discussed in great- 
er detail in the sections below. 

Pyogr^S That Develop Abili ties Usiny Generic Materials 
The two programs reviewed in this section work toward an overall goal of 
developing particular thinking skills through the use of generic or conte.nt- 
free materials. It is thought that this allows students to focus more spe- 
cifically on skill development without the student's prior knowledge level in- 
terfering with efforts to develop skills. It is also assumed that successful 
improvement of target skills will result in students becoming better thinkers. 
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Table 3: Overview of Thinking Skills Emphasized in Programs with Single Goal 



Group 1: Programs with 
Single Goal 


goal A: CQQOitive Abilities Develoosd as Primary Pf^ram Goals 


Structure of the Intellect 
(SOI) 


120 separable intelledual skills are derived from crossina five mental ooeratronc. (nnr|nition m«mnry ..oinofj^n ^nv-rnr nt 
production, divergent production) with tour contents (figural. symbolic, semantic and behavior) and six oroductlons (um^ 
classes, relations, systems, transformations, implications). 


Cognitive Research 
Trust (CoRT) 


Pgrggptual tools taught by generating ideas, reflecting on perceptions, organizing both, prioritizing, considering other points of 
view. Focuses on anaiytk: and k>gical skills needed for reflecting on perceptions. Creative thinking. 


ThinkAbout 


13 Basic Reasoning Skj|[g in finHIng aitQmotjyf^o r^llactinq information, finding onttorn:: cla<L<;Tfvinn nqinn rritAria rochonmn 
information, judging information. £5 sMCifiC Skills in math, language arts, study skills. Tips and challenges" on memory 
improvement, goal setting. r ^ j 


Sclence..,A Process 
Approach (SAPA) 


Learn processes involved in scientific work: process as an intellectual skill, especially as it relates to processing information. 



Stprv(?t;ur^ pf In^^UfrPt program. The structure of Intellect Program was de- 
veloped by the SOI Institute, a nonprofit educational corporation located in 
California. The institute provides tests, diagnostic services, prescriptions 
of exercises designed to address weaknesses identified by the tests, and exer- 
cises to carry out the recommendations (Nickerson, Perkins, & Smith, 1985). 
Since evaluation activities have included students in all elementary grades 
through eighth grade, it is assumed that is the target student population. The 
SOI Program (see Table 3) is based on the idea of remediation of deficiencies 
in abilities. Remediation decisions and prescriptions are based on Guilford's 
(1967) structure-of- intellect theory. This theory assumes that a total of 120 
separable intellectual skills can be derived from crossing five mental op- 
erations --cognition, memory, evaluation, convergent production, and divergent 
production; with four contents --figural, symbolic, semantic, and behavior; and 
six products --units, classes, relations, systems, transformations, and implica- 
tions (Sternberg & Bhana, 1986). Support for this theory, offered in the form 
of factor analysis, has been criticized for its statistical techniques, but 
still remains the foundation upon which the program is based (Nickerson et al., 
1985). 

Other program features are summarized in Table 2 and discussed below. Ma- 
terials in the program are developed most extensively for 25 of the 120 intel- 
lectual skills, which are considered most relevant to school subject areas. De- 
ficiencies in intellectxial skills are Identified before the program begins 
through the use of the Structure of Intallect-Leaming Abilities test (SOI-LA). 
Exercises are prescribed for deficient areas (as identified by the test) , and 
students then take a posttest. Because of the individually prescribed nature 
of the program, students will experience different aspects of what the program 
has to offer (Sternberg & Bhana, 1986). The exercises are test-like in nature, 



and are content-free in that the object of thought and focus is on developing 
the skill, not the content. In addition, the materials are "teacher proof in 
the sense that the test (SOI-U) makes decisions about prescription (not the 
teacher), and the training program for teachers is "relatively regimented" 
(Sternberg & Bhana, 1986, p. 65). Thus, the teacher's role is reduced to 
implementation, with little variation expected in the way particular training 
exercises are implemented across students. 

Studies on the effectiveness of the SOI program are limited to those car- 
ried out by the SOI Institute. They address three issues: (a) the reliability 
and validity of SOI instruments, (b) the extent to which training produces in- 
tellectual growth, and (c) the impact of SOI instruction on school performance 
and other work (Nickerson et al., 1985). Studies in the first area show 
problems with statistical reliability and show different factors as important 
across studies. Studies in the second area show gains, but the intellectual 
growth is measured by SOI instruments, which raises the issue as to whether the 
program teaches to the test, especially since the exercises are so similar in 
format to test items. Only two studies are available for the third area. One 
showed student gains in mathematics but not reading. The second showed gains 
in 18 of 26 factors addressed, but there was no control group. The transfer 
that is indicated seems to apply more to transfer to other tests that measure 
skills similar to those measured by SOI instruments. 

Thus, there are some indications of gains in intellectual growth and gains 
in application of abilities to school subjects, but the studies are not clearly 
reported, nor are the designs of the studies adequate (Sternberg & Bhana, 
1986). No information is provided as to whether students enjoy or are chal- 
lenged by the exercises. Particularly for use with young children, the format 
(completing test-like instruments with little or no interaction) may lack 
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stimulation. There are no direct efforts made by the teacher to encourage 
transfer to school subject areas or other situations, which greatly reduces the 
likelihood of transfer. 

Cognitive Research Trust. Cognitive Research Trust (CoRT) was developed 
by Edward deBono for use with people of all ages spanning ages five years 
through adult. Its commercially available materials are widely used in more 
than 5,000 schools in Venezuela, England, Scotland, Wales, Eire, Australia, New 
Zealand, Canada, Spain, Malta, and Nigeria (deBono, 1985). The program rests 
on the assumption that of the two basic thinking stages, perception and 
analysis, lack of precise perception is most often the cause of poor thinking 
(deBono, 1985). DeBono further claims that people do not generally lack percep- 
tual skills. Instead, they simply are not aware of their importance, are not 
used to using them, and therefore do not. Two main purposes of the exercises 
and activities in the program (see Table 3) are to help people learn simple 
tools for directing their attention appropriately in thinking and to provide 
ways for them to practice using the tools so their use becomes automatic 
(Chance, 1986). Transfer of the tools is expected to apply more specifically 
to thinking in everyday situations than to school tasks, although deBono (1985) 
also claims that developing automaticity in using the tools will eventually 
promote clearer thinking in all areas. 

Table 2 includes a summary of additional program features. Materials in 
the program for people age 12 and over are divided into six units (Breadth, 
Organization, Interaction, Creativity, Information and Feeling, and Action) . 
Materials are content- free, based on the assumption that it is helpful to stu- 
dents to remove content knowledge as a potential obstacle in the thinking pro- 
cess so that they can concentrate more fully on developing perceptual tools. 
These materials have been used for children slightly younger (ages 9 through 
11) if their verbal skills are sufficient to benefit from the materials. 

11 



A "Thinking Course for Juniors" is available for children ages 5 through 
12. The course for younger children ufas visual items to compensate for the 
lesser verbal skills expected of this age group, and centers around designs or 
inventions. Children draw designs (e.g., a chocolate -making machine), make par- 
tial drawings, criticize completed drawings, or illustrate situations in a car- 
toon format. The exercises are provided to practice thinking, with no direct 
instruction in thinking provided (Poison & Jeffries, 1985). In addition, there 
is a collection of games called Think Links intended to facilitate thinking 
practice. The games vary for different age groups. 

The 6 units in the regular CoRT program consist of 60 lessons. The pro- 
gram is used most often with children spanning the ages 9 through 12 (Chance, 
1986). Presumably the format for conducting the lessons is similar to that in 
the Thinking for Juniors version. Typically, students will complete one CoRT 
lesson weekly that lasts between 30 and 50 minutes. According to Chance (1986), 
each lesson follows the sequence: Introduction (describing the tool to be learned), 
Practice (small group practice using the tool). Process (class discussion of 
the tool). Principles (group work designed to focus on the nature of the tool), 
and Project (additional practice items that are completed if time permits). 

The teacher's role in the learning process is to facilitate practice using 
the perceptual tools. For example, CoRT teachers are directed to keep the dis- 
cussion narrowly focused on one tool at a time. In addition, they are directed 
not to review tools previously practiced during a lesson on a new tool. This 
is based on the assumption that such overlap will just confuse students, and 
that purposeful practice of one skill at a time is most likely to develop the 
desired automaticity in perception that is needed for better thinking (Chance, 
1986). Teachers may use their judgement to vary activities and include role 
playing, dramatization, drawing and writing, but they are cautioned to keep the 
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focus cn the tool being practiced. Like SOI, program materials are highly 
structured, and it is assumed that teachers will be highly motivated to use the 
materials provided to learn about the program content and optimal ways to imple- 
ment it. 

Despite the program's use for over 10 years, there is no published informa- 
tion regarding its effectiveness, and only deBono's informal research methods 
have been used to assess the effectiveness of the six instructional units in 
the regular CoRT program- -information observations, informal studies incorpo- 
rated into an instructional program, unpublished notes (see Poison & Jeffries, 
1985). In addition, many of the populations for studies conducted vary on the 
extent to which the program materials were actually used with students. Never- 
theless, results show transfer of those performances in which students were spe- 
cifically trained to other tasks similar to those in the CoRT activities. In 
addition, students do typically generate more ideas regarding issues that 
require abstraction and a balanced view of problems. These findings do not, 
however, demonstrate transfer to activities less similar to CoRT exercises, nor 
do they show transfer to thinking in school subject areas (Nickerson et al., 
1985). 

Others who have reviewed the CoRT program reveal additional problems. For 
example. Poison and Jeffries (1985) point out that CoRT materials contain a 
great deal of overlap across thinking tools to be developed. For instance, 
goal formation is covered in three lessons (Define the Problem, Target, and 
purpose) . Since deBono explicitly directs teachers not to disccss ways in 
which tools across lessons interrelate, this might confuse students, and does 
not provide guidance for them to interpret how and when to use the apparently 
separate tools in everyday contexts. Chance (1986) notes that student and 
teacher response to the program is. not always positive. One objection is the 



use of acronyms. (e.g., PMI, CAF, OPV) to name different tools and another that 
students often feel they already know how to use the tools they are practic- 
ing. DeBono's counterargument is that students may know them, but they have 
not learnsd to use them. Finally, the regimented pattern that the lessons 
follow becomes monotonous for the students. DeBono acknowledges this problem, 
but instead of restructuring the lessons, he recommends increasing the tight 
focus on the tool during each lesson. This does not seem a likely relief from 
monotony. 

Pyogramg That Develop Abilitie s Using Snhiect Matter-Related MatP.ripl^ 
Two programs are reviewed in this section: ThinkAbout and Science ... A 
Process Approach. Each of these programs identifies developing thinking skills 
as an important outcome, and each uses subject matter- related materials as a 
means to develop specific abilities. Through the use of such materials, there 
is more direct emphasis on helping students see how thinking skills apply to 
school subjects than with the SOI and CoRT programs that use content- free mate- 
rials. However, these programs clearly place a lower priority on subject mat- 
ter learning than on skill development. Unlike SOI and CoRT where the degree 
of teacher mediation is minimal, these two programs place greater importance on 
the role of dialogue and interaction in the learning process, so the teacher 
has a central role in helping students develop abilities. 

ThinkAbout 

The goal of the ThinkAbout Program is to develop independent learners and 
problem solvers, it was developed from 1976-78 for fifth-and sixth-grade stu- 
dents by the Skills Essential to Learning Project, a consortium of over 40 
American and Canadian state and provincial agencies (Nickerson et al., 1985). 
It is assumed that students will develop effective problem solving skills by 
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seeing and discussing how others solve problems and by being motivated co en- 
gage in problem-solving activities. 

The program identifies thirteen basic reasoning skills (e.g., finding al- 
ternatives, collecting information, finding patterns, classifying, using crite- 
ria, reshaping information, judging information; see Table 3). A total of 
sixty 15-minute video programs focus on the 13 basic skills, devoting about cwo 
to six programs to each skill. The program also identifies about 65 specific 
skills in mathematics, language arts, and study skills which the curriculum ma- 
terials are designed to reinforce. Nine categories organize the 65 skills: 
reading; writing; listening; speaking/discussion/presenting; viewing and observ- 
ing; graphs, maps, and scale drawings; measurement; computation; and study 
skills. Finally, a third emphasis is on "tips and challenges" designed to fos- 
ter such things as memory improvement, goal setting, and handling anxiety 
(Nickerson et al., 1985). 

Table 2 includes a summary of additional program features. The teacher's 
guide is extensive and outlines the overall purposes, rationale, content of the 
videotape, ways to prepare students for viewing the videotape, and specific sug- 
gestions for follow-up discussions and activities. Thus, discussion and inter- 
action surrounding the problem-solving process is an important part of the pro- 
gram, and adequate teacher preparation is the key to successful implemencacion. 
The content of the videotapes blends posing problem-solving situations that ma- 
he related to school tasks (e.g., having to write a three-page biographical re- 
port on George Washington Carver) with a focus on a skill cluster (e.g., learn- 
ing how to find alternatives in a problem-solving situation; see Nickerson ec 
al., 1985). It is unclear as to how the program developers intend the student:: 
to connect the specific content- related skills with the 13 basic reasoning 
skills, or how they provide support for such connftctions to be made. 



15 

23 



One evaluation of the first year of use as an entire series was conducted 
during the 1979-80 school year (Nickerson et al.. 1985). Surveys, case studies 
and objective tests were administered in 241 classrooms. Standardized test 
scores show no clear differences in performance between students who did and 
did not participate in the program. In addition, implementation problc..^ were 
identified. There were great differences across classrooms in how the 
ThinkAbout Program was tied to the rest of the curriculum and in how the teach- 
ers prepared to use it. Students sometimes perceived the use of television as 
a break from work instead of focusing on the goals of the program (to develop 
problem-solving abilities and become independent thinkers). Survey results 
showed that students and teachers generally liked the program, and teachers 
felt students' abilities improved. Thus it seems that the program may have 
been more successful in providing an enjoyable context for solving problems and 
less successful in helping students develop specific skills or in helping them 
apply those skills to school subjects. 

Scienc e ... A Process Approach 

Science ... A Process Approach (SAPA) was developed for use with grades 
K through 6 by the Commission on Science Education of the American Association 
for the Advancement of Science. The main goal of the program is to help stu- 
dents learn the processes involved in scientific work (see Table 3) . Underly- 
ing the goal of teaching scientific processes is Gagne's distinction among 
three types of processes: (a) process as distinguished from content; 

(b) process as an activity in which a scientist normally engages; and 

(c) process as an intellectual skill, especially as it relates to processing 
information (cited in Nickerson et al.. 1985). The third process is of 
particular interest in this review. 
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Scientific processes are organized in a behavioral hierarchy that is cat- 
egorized into five stages within each grade level across grades K through 6. 
Table 2 outlines main features of the program. There are 105 modulas that can 
each be taught in on^.^ class period. Materials and teacher instruction are pro- 
vided for each module. As in ThinkAbout, the teacher materials outline objec- 
tives, rationale, instructional procedures, and methods. In the methods sec- 
tion, optional activities are provided to giva the teacher maximum flexibility 
in deciding the best way to help students develop their use of the proccf.ses. 
In addition, the program provides supplementary materials for the teacher which 
includes a Progyam Gujd^ (strategy descriptions) and Commentary for Tear.her5; 
(5elf- instructional units on the processes of science). Presumably, the 
teacher will rely on the program materials for training. Implicit in the 
"hands-on" approach to scientific investigation (e.g., learning to observe by 
having the opportunity to observe and reflect on the experience) is the notion 
that, like teachers using ThinkAbout, teachers will play an integral role in 
helping students develop the skills needed to carry out scientific processes. 

Instead of assuming transfer will occur through using materials that apply 
to a variety of contexts- -as ThinkAbout developers assume- -SAPA developers spe- 
cifically direct teachers to give explicit attention to transfer. Each module 
includes "generalizing experiences" that require students to apply who., they 
have learned in that module to other contexts (Nickerson et al., 1985). Some 
modules also provide instruction in mathematics and language skills, and th:5 
developers have as a clear goal that the effects of learning scientific pro- 
cesses will generalize to other subject areas (Gagne cited in Nickerson et al., 
1985). 

A series of experiments was conducted on the implementation of SAPA with 
fourth-and fifth-grade students. Results show that SAPA modules can show 
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gains, providing student progress is measured throughout instruction and fur- 
ther instruction is provided as needed (Nickerson et al., 1985). No transfer 
studies have been conducted to investigate the developers' claims of generaliz- 
ability of effects of learning scientific processes to other domains over time. 

Summary 

The four programs reviewed in this section share an overall goal. They in 
tend to teach skills that will improve thinking. There is some evidence to sup 
port the claim that they do have some short- term positive effects in the areas 
they target. There is not enough evidence, however, to predict the extent to 
which positive effects will make a difference in students' abilities to think 
long term or in their abilities to think about subject matter domains or every- 
day problems. Those issues clearly need further study. 

Nevertheless, examination of the programs' similarities and differences re- 
veal some lessons learned from these programs (see Table 2) . Programs that ad- 
dress the issue of transfer of learning to other domains explicitly as part of 
their instruction (e.g., SAPA through "generalizing experiences") may have a 
better chance of succeeding than those that rely on transfer occurring because 
of the variety of contexts included in the materials (e.g., ThinkAbout) . It ap- 
pears that helping students become aware of the need for transfer and instances 
where transfer apply increases the likelihood that they would make such connec- 
tions, compared to hoping the connections will be made (Sternberg, 1987). How- 
ever, more information is needed to support this claim. 

A second lesson to be learned is that the programs that use content- re- 
lated materials (e.g., ThinkAbout, SAPA) seem to be more interesting to stu- 
dents and seem to do a better job helping students develop desired dispositions 
to use thinking skills. SOI and CoRT exercises are more test-like in nature, 
so that, even though students may become engaged in solving specific problems 
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posed in the exercises, there is less likelihood that they will generalize the', 
such engagement applies to other contexts. 

A third lesson is that active engagement in problem-solving and thinking 
activities, does make a uifference. and teachers play an important role in the 
process. SOI has not been very well received by students which may be due to 
its lack of emphasis on the role of the teacher. CoRT teachers are cautioned 
to limit the focus of discussion on perceptual skill development, and therefore 
bypass the opportunity to help students see ways in which skills can be used 
across experiences. The CoRT program is criticized by students as being 
monotonous, boring, and geared toward teaching skills students feel they al- 
ready know. This may be partially due to the use of generic materials, or it 
may be due to the narrow focus of the discussions surrounding the materials. 
ThinkAbout and SAPA program developers see active discussion and reflection on 
the thinking process as key to the successes they claim for their program. 

Finally, while all four programs claim that teacher preparation for using 
the program is important, they devote little energy (beyond providing detailed 
materials) to specific efforts to train teachers. Evaluation efforts have illu- 
minated the trem*endous difficulty involved in assessing student learning in the 
area of developing thinking skills. The same complexity exists in teaching 
such skills, and teachers should not be expected to be experts in teaching 
thinking simply by reading program materials. Perhaps if teachers were better 
trained in the underlying assumptions of the program, in carrying out effective 
implementation (e.g., thoughtful and lively discussions, diagnosing when stu- 
dents need further help) , and in what to look for as evidence of student 
progress, some of the programs would provide better evidence to support their 
claims. 
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Programs With Multiple Goals 
Eight programs are reviewed in this section: the Instrumental Enrichment 
Program (IE), the Productive Thinking Program, Odyssey, Talents Unlimited (TU) , 
Philosophy for Children, Reciprocal Teaching, the Expository Writing Program 
(EWP), and LOGO. Like the first four programs reviewed, they share as a major 
goal a focus on developing specific skills. In addition to that goal, however, 
these programs take on one or more of the following goals: developing 
metacognitive awareness and strategies; developing conditional knowledge about 
contexts for knowledge and skill use; and developing attitudes and dispositions 
to use the skills (see Table 4 for a summary of each program's goals). 

These eight programs are further divided into three groups according to 
the methods and materials they use. One group includes those programs with mul- 
tiple goals that use generic materials (IE and the Productive Thinking Pro- 
gram) . A second group includes three programs that focus on helping students 
apply the four broad program goals in an integrated fashion (Odyssey, TU, and 
Philosophy for Children), A third group includes those programs that help stu- 
dents work toward program goals through language- -reading and writing- -and 
symbol manipulation- -mathematics (Nickerson et al., 1985). ' These programs are 
Reciprocal Teaching (reading comprehension) , EWP (written composition) , and 
LOGO (computer programming;. A summary of these prograias' characteristics is 
shown in Table 2, and discussion of ways in which these characteristics 
increase the likelihood of transfer of skills taught is included in the sec- 
tions that follow. 

Programs With Mult iple Goals That Use Generic Materials 

IE and the Productive Thinking Program are based on the assumption that di- 
rect and systematic training in the use of particular skills will improve think- 
ing. They also share the assumption (along with SOI and CoRT) that it is 
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helpful to remove content knowledge as a potential obstacle in developing the 
thinking processes. The developers assume the use of generic materials helps 
students concentrate more fully on developing skills and strategies that will 
help them think better. Thus, the emphasis in the program is on process, not 
content. In both programs, the teacher plays an active role in the learning 
process. In this way they contrast with SOI and CoRT which give a fairly 
minimal role to the teacher. 

The Instrumental Enrichment Program, The Instrumental Enrichment Program 
(IE) was developed by Reuven Feuerstein for use with "low- functioning" adoles- 
cents between the ages of 10 to 18 years. The term "low functioning" refers to 
an individual's lack of capacity to respond to new experiences. Thus, the goal 
of the program is to help socioculturally deprived adolescents (approximately 
age 11 on up) change their characteristic ways of dealing with information so 
that they are more receptive and sensitive to internal and external sources of 
stimuli, and thereby more readily able to learn from stimuli (see Table 4, Goal 
A). Presently, the program is used in Israel, the United States, Canada, and 
Venezuela (Nickerson et al., 1985). 

A major assumption on which the program is built is that children learn 
how mature thinkers think by participating in interactions with adults, re- 
ferred to as mediated learning experiences (MLE) . Through everyday inter- 
actions with adults where there is a sufficient amount of social interaction 
surrounding experiences, children learn how to link, organize, relate, and 
transform relationships and also learn how to use various modalities of think- 
ing (e.g., representational, inferential, hypothetical). Feuerstein h)rpoth- 
esizes that children who have not had these types of mediational learning expe- 
riences as part of their development are less able to understand academic mate- 
rial or learn new content because they do not perceive new content 



analytically, nor do they elaborate new information and relate it to prior 
knowledge (Pressley, Cariglia, & Snyder, 1985). 

Feuerstein further hypothesizes that deficiencies resulting from lack of 
mediated learning experiences can be remediated by appropriate exercises and 
that learners can therefore increase their capacity to learn after such re- 
mediation. The use of specific exercises is similar to methods used in SOI and 
CoRT to improve the capacity to think. Table 2 summarizes additional program 
features. Feuerstein developed a set of 15 exercises or instruments designed 
to provide a context for mediated learning. These paper-and-pencil exercises, 
ordered in increasing difficulty, are intended to expose the learner to cogni- 
tive skills used by effective learners, and to develop general concepts, 
skills, and attitudes that will make it possible for learners to learn more ef- 
ficiently on their own(see Table 4, Goals B & D) . 

The teacher's role is to provide support by encouraging appropriate cogni- 
tive activities relevant to the materials. This support and interaction sur- 
rounding the experience may come during the introduction, the individual work 
time, class discussion time, or summary phase of the exercise. Ordinary teach- 
ers, with extensive training (at minimum a two-week workshop), can learn to use 
the materials and can only obtain the materials if they participate in the 
training. In addition, Feuerstein values the quality of the interaction be- 
tween teachers and students more than the exercises themselves (Chance, 1986), 
which sharply contrasts with the teacher's role in SOI and CoRT, where the 
teacher merely implements program exercises. 

The materials do not presuppose sophisticated content knowledge so that 
students can concentrate on the learning process. The object of thought is 
learning to see different kinds of relationships and connections, but not 
through subject matter-specific materials. Thus, exercises may include such 
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"content -free" tasks as connecting dots so that they conform to a model or iden- 
tifying how everyday objects are oriented to one another on a drawing 
(Feuerstein, Jensen, Hoffman. & Rand. . 1985). Students are to spend three to 
five hours per week completing the program over a two-year period. 

Once students have worked through the set of 7.5 instr-tments , what can one 
expect about their transfer of skills they are developing through the exercises 
to their overall ability to learn? Two aspects of the program attempt to ad- 
dress this issue. First, each lesson focuses on one or two principles that 
have broad generality so that each lesson has a focal theme that can be related 
to themes in subsequent lessons (e.g., "when cues diminish, we have to change 
strategies"), and students are encouraged to relate one experience to another 
across the exercises (Bransford, Arbitman-Smith, Stein, & Vye, 1985). Again, 
this contrasts with CoRT where the teacher is discouraged from discussing with 
students how exercises in one area overlap, or connect with others. Second, stu- 
dents are encouraged to "bridge" the principles brought out through the exer- 
cises to ways in which the principles apply to their own experiences (Table 4, 
Goal B). 

These two aspects are dependent on the teacher mediating or supporting 
their occurrence. It is believed that finding examples of application of the 
principles across the exercises and in everyday experience will enccarage trans- 
fer. The exercises themselves are invented with the intention of being "intrin- 
sically interesting," thereby offering the opportunity for students to feel the 
pleasure of succeeding at accomplishing something that is inherently interest- 
ing. This addresses a secondary goal of the program, to develop attitudes and 
dispositions conducive to using thinking skills and enjoying their use (Table 
4, Goal D). It is hoped that such experiences will provide the tools, confi- 
dence, and motivation for students to learn to solve problems independently 
(Bransford et al., 1985). 
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Feuerstein provides rich examples of individual cases where the IE Program 
has shown dramatic results (Chance, 1986). However, in a review of 38 studies 
of the program, Sternberg and Bhana (1986) report mixed results. Evaluation 
difficulties included variation in the subject populations (country, social 
class, age, intellectual level, educational level), amount of exposure to the 
program, and the extent to which particular instruments were actually used. In 
addition, the evaluations focused more on measuring academic gains and provide 
little information about development of insight and motivation, which is the 
secondary goal of the program. Other concerns included the similarities 
between the problems in the exercises and testing devices used, so teaching to 
the test must be questioned. Finally, since the program requires a separate 
curriculum of long duration, extensive time spent each week, and extensive 
teacher training, they were also concerned about a ••justification of effort" 
effect on the evaluation efforts (Sternberg & Bhana, 1986, p. 63). 

Sternberg and Bhana (1986) do report some positive findings from their re- 
view. There do appear to be gains attained on standard types of IQ and apti- 
tude measures if the full program is administered by well trained, motivated in- 
structors. Abstract reasoning and spatial visualization are areas that are 
more likely to show gains than verbal skills. They caution that such gains are 
not likely to be dramatic, since it is unrealistic to see dramatic gains in 
such areas over a short time period. They found no evidence of gains in 
insightful, creative, or sjmthetic thinking abilities. 

They report that transfer to school work is likely -in some cases," but do 
not elaborate on the issue. They note that transfer does seem to depend on the 
extent to which teachers successfully -bridge- the principles in the exercises 
with other experiences requiring the same principles. Similarly, Nickerson et 
al. (1985) report on Arbitman- Smith, Haywood, and Bransford's 1984 evaluation 



which revealed that transfer of ability to solve problems does occur in 
domain- specific and domain- independent problems, but they do not elaborate on 
their findings. 

Finally, there is the question as to whether the gains noted in individual 
cases can be adequately assessed by the standardized intelligence tests used in 
most evaluations (Nickerson et al., 1985). Reports of dramatically changing 
low- functioning adolescents' options in later life, or the enthusiastic accep- 
tance of using the exercises do not seem to be captured in evaluations as they 
have been carried out (Chance, 1986). 

In another extensive review of reports on empirical research on the IE Pro- 
gram, Savell, Twohig, and Rachford (1986) express similar concerns with Inter- 
preting the research reports. They note problems with research design and 
implementation and the sufficiency of information provided about the study 
(especially describing the extent to which and how the program was used). 
These problems contribute to confusion concerning how to interpret research 
reports. Nevertheless, they identify a subset of studies conducted in Israel, 
Venezuela, Canada, and the United States tha reveal some generalizations and 
conclusions that seem reasonable. For example, statistically significant group 
differences were found in populations in these four countries on certain 
standard nonverbal measures of intelligence. Instruments used measured skill 
in processing (figural and spatial information), but did not provide clear or 
consistent information about gains in areas such as self-esteem, impulsivity, 
classroom behavior, academic achievement, or course content. 

Studies that showed these gains had several things in common regarding pro- 
gram implementation: (a) Instructors had at least a week of training, (b) expo- 
sure to the program was generally 80 hours or more ove; a two-year period, and 
(c) the program was taught in conjunction with some other subject matter that 
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was interesting and important to the students. Thus, this review makes the im- 
portant point that the way the program is implemented apparently has a great 
deal to do with the internal success shown. Moreover, among several sugges- 
tions for improvement in studies that would make results more interpretable is 
the suggestion to focus more specifically on studying the possible effects of 
the "bridging'* exercises, which would reveal more information about transfer. 

Productive Thinking Program. Martin Covington's Productive Thinking Pro- 
gram was developed for upper elementary students out of a perceived need to 
find ways to combat self-defeating achievement patterns in schools. Covington 
argues that the nature of reward structures in classrooms works against 
students' motivation to pursue success. In addition to modifying reward 
systems in classrooms, teachers can teach students component skills (Table 4, 
Goal A) involved in strategic thinking (Table 4, Goal B) so students can take 
advantage of newly created opportunities for classroom achievement. He defines 
strategic thinking as •'the capacity to identify and analyze problems and to 
create and monitor plans for their solution** (Covington, 1985, p. 390). Thus, 
major goals for the program include helping students develop heuristics for 
generating and developing ideas and for learning to use them in an overall 
organized approach (Table 4, Goal 3). Secondary goals important to the program 
are that students should value and enjoy thinking activity, and develop the 
propensity to persist when searching for solutions to problems (Table 4, Goal 
D) (Nickerson et al., 1985). The program has been used in a wide variety of 
settings, and of programs designed for use in the 1960s and 1970s, this one has 
been the most tested (Nickerson et al., 1985). 

Table 2 summarizes additional program features. The program consists of 
15 booklets centered around developing awareness and use of five kinds of think- 
ing skills: discovering and formulating problems, organizing and using 
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inforaation. generating ideas, evaluating and improving ideas, and creating new 
perspectives. Each booklat contains one basic lesson and one problem set. al- 
though the materials are not systematic about which type of skill is being 
taught. Each basic lesson tells a story in cartoon format. The two main char- 
acters. Jim and Lila. are children about the ages of the target audience. In 
each lesson, they solve a mystery with the help of their Uncle John, a high 
school Science teacher. 

Students are drawn into actively participating in the mystery-solving pro- 
cess by being asked to generate ideas of their own as the lesson progresses. 
For example, when Lila and Jim must repeatedly tackle a problem, students are 
also asked to write their ideas in a Reply Notebook where they state problems 
in their own words, generate ideas, or examine facts. The teacher gives feed- 
back each step of the way so students can evaluate their progress throughout 
the problem-solving process. Following each basic lesson, students complete 
one problem set. These are intended to provide additional practice in the 
thinking guides included in the lesson. Instead of centering around Jim and 
Lila's problems, the content of the pi^ctice problems involves real problems 
taken from the social and natural sciences, history, and human relations 
(Chance. 1986), 

The basic lessons and problem sets can be done individually without group 
discussions, but: the program developers recommend more teacher guidance. Dur- 
ing each part of the problem- solving process, the teacher is encouraged to give 
feedback about student performance as well as to encourage students and monitor 
the students' involvement and interest level. In other words, the teacher's 
role goes beyond facilitating the thinking process to taking on the additional 
goal of helping students develop the appropriate attitudes and dispositions to 
value and use che processes. This requires lively discussion, as in two of the 
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programs reviewed thus far (e.g., ThinkAbout, SAPA) . This kind of teacher in- 
volvement (convergent and divergent focus in discussions, close monitoring of 
students progress, attention to student attitudes and dispositions) contrasts 
with the narrowly focused involvement called for in CoRT exercises. Like CoRT, 
SAPA, ThinkAbout and SOI, the Productive Thinking Progran: provides guidance to 
teachers through its materials and does not assume further training is neces- 
sary. However, it is assumed that teachers have the motivation and desire to 
study closely the materials and pay attention to appropriate implementation. 

Several evaluations have been conducted on the Productive Thinking Program 
and provide evidence that when test measures are similar to program exercises 
(problem-solving situations in which subjects are gradually led through the 
thinking process), performance is enhanced (Nickerson et al., 1985). However, 
there may be some major difficulties with interpreting some study results be- 
cause of dependent measures used and the way they were scored (Poison & 
Jeffries, 1985). For example, the measures include a wide variety of problem- 
solving experiences that are quite open ended and therefore without a particu- 
lar solution. In addition, many items were scored for characteristics such as 
number of ideas generated, but there were no criteria set for determining the 
quality of ideas nor for how the number of ideas generated influenced reaching 
a problem solution. 

Other studies showed improvements in students' willingness to tackle com- 
plex problems, to value their own opinions even if they differed from the 
group's, and students showed more confidence in their own overall abilities as 
thinkers (Poison & Jeffries, 1985). On the whole, results show improvements in 
solving specific types of problems, but do not show evidence of transfer to 
solving more complex extended problems independently. This may be due to the 
specific nature of the exercises, which are designed to lead the students step 
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by step through the thinking process to a specific solution. Such exercises do 
not provide guidance or practice in solving aore complex problems that 'nay not 
lead to one solution (Nickerson at al., 1985). 

An additional problem raised regarding the program materials i.<; that they 
do not consistently treat particular thinking skills. Instead, there is a 
great deal of skills overlap across the various exercises, and sometices stu- 
dents are asked to use skills that have not been formally introduced. %ile it 
can be argued that such overlap may be helpful in having students experience 
the complex nature of the problem solving process, there is the potential prob- 
lem of not teaching them how to bring together appropriately the specific ,T.sta- 
cognitive and problem-solving skills. Moreover, the exercises tend to give ov- 
erall guidelines and advice on using various thinking skills (e.g., think of un- 
usual ideas), but do not elaborate on how to go about doing so, or how to de- 
cide when one is using the advice appropriately (Chance, 1986; Poison & 
Jeffries, 1985). 

Programs That Focus on Inte grated Application of ana^<i 
Odyssey, Talents Unlimited, and Philosophy for Children use content- 
related Qaterials to help students learn to develop and apply skills in an 
integrated fashion. That is, specific attention is given to each of four broad 
program goals (see Table 4) and students are asked to apply particular skills 
(Goal A) and strategies (Goal B) taught in contexts that help them understand 
how and when to use them (Goal C) . In addition, there is an emphasis on devel- 
oping the values and disposition to use the skills in learning and everyday 
situations (Goal D) . 

Odvssev 

Odyssey (also called Project Intelligence) was developed by an interna- 
tional team of American and Venezuelan psychologists, educators, and 
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governmental officials (Chance, 1986; Nickerson et al., 1985). It was designed 
for students of all abilities in fourth through sixth grades, but actual use 
has been wich Venezuelan seventh graders during the 1981-83 school years. Mate- 
rials are written in English and Spanish. The program's goal is to help stu- 
dents develop the ability to perform a wide variety of intellectual tasks, so 
it focuses on helping them develop "target abilities" in six areas: foundations 
of reasoning, understanding language, verbal reasoning, problem solving, deci- 
sion making, and inventive thinking (Table 4, Goal A). Tne program materials 
are "content rich" to maximize the variety of contexts in which the target 
abilities might be used, which increases the likelihood that the skills will be 
accessed in new situations (Adams, 1986). 

The main goal of the program is to help students develop abilities, but as 
Chance (1986) points out, program developers also believe that skilled intellec- 
tual performance requires learning in three other areas as well (see Table 4, 
Goals B, C, D): methods (appropriate ways of approaching tasks), knowledge 
(facts, concepts, principles used in the thinking process), and attitudes 
(point of view, perspective that enhances performance). Therefore, discussion 
focuses on helping students learn when and how to apply target skills in addi- 
tion to providing opportunities to practice them. When new knowledge is needed 
to complete exercises, it is taught. It is assumed that the content-rich mate- 
rials will interest and motivate students to help them experience and enjoy the 
rewards of b«ing curious, showing respect for others' opinions, and enthusiasm 
for learning (Chance 19CS). 

Table 2 provides a summary of other program characteristics. The program 
is divided into six lesson series (according to the six target area abilities 
listed above) . For each of these six categories there is a student book and 
teacher's aanual for a total of 99 individual lessons. Each 45-minute lesson 
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focuses on a specific set of instructional objectives. The teacher's manual 
spells out the following for each lesson: lesson format, rationale, objec- 
tives, target abilities, materials, classroom procedure. Classroom procedure 
information includes detailed scripts or imaginary dialogues that illustrate 
for teachers how they might facilitate the dialogue for the lesson. They are 
intended to aid teachers in understanding the nature of dialogue that is consid- 
ered to be an essential part of the program, and such dialogue includes a focus 
on how and why target abilities apply to a particular situation (Chance, 1986; 
Nickerson et ai., 1985). Formal teacher training is offered by the publisher, 
but the details in the teacher's manual for each lesson could be adequate for 
teachers motivated to use their guidance fully (Chance, 1986). 

The materials cover a vast range of topics. Some are more general, such 
as having students figure out the nature of changes they see in a series of 
drawings (and focusing on which aspects one should look for when considering 
change). Others pertain to everyday situations such as speculating on authors- 
intentions in writing a business letter, a personal letter, a newspaper ar- 
ticle, or a restaurant ad. Some exercises use materials common to school as- 
signments such as asking students to read a fable and select a moral from among 
four alternatives (Change, 1986). This range is purposely provided as a way to 
maximize the applications of target abilities across a wide range of situations 
in hopes that transfer is more likely to occur (Adams, 1986). 

Two forms of evaluation have taken place with the implementation of the Od- 
yssey Program in Venezuela. One formative evaluation was conducted in 1981-82 
to help identify strengths and weaknesses while the program was still being de- 
veloped. A summative evaluation was conducted in 1982-83 when the program was 
implemented in barrio schools (students were from families with low socioeco- 
nomic status and minimal parent education) (Nickerson et al., 1985). When 
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comparing a control group with the experiaental group, gains were found in both 
groups in tests of target abilities and standard tests, but gains in the ex- 
perimental group were largsr, especially in target abilities. These prelim- 
inary results point to greac promise in the program (Chance, 1986; Nickerson et 
al., 1985; Sternberg & Bhana, 1986), but more needs to be learned about the ex- 
tent to which such success would occur with other student populations. A spe- 
cial posttest, given to a smaller sample of students, also showed improvements 
in the extent to which skills taught (e.g., appropriateness of design, clarity 
of expression, use of supporting reasons) transfer to educational and practical 
tasks (Resnick, 1987). 

Talents TTnlimited 

Talents Unlimited (TU) is an inservice education model designed to help 
teachers nurture children's thinking skills, which in turn is expected to im- 
prove children's academic performance. It was developed under a Title III 
Elementary and Secondary Education Act grant in Mobile, Alabama, in 1971. 
Since that time, its use as an inservice model and implementation in classrooms 
has spread to over 800 sites across the nation (Hobbs, 1988). Additionally, a 
growing number of uecondary schools are also adapting the model for use in 
middle and high schools, so developers are working toward a creating comprehen- 
sive K-12 model (Hobbs, 1988; Schlichter, Hobbs, & Crump, 1988). 

Like SOI, TU is derived from Guilford' 3 structure of the intellect model 
of intelligence, but ualvin Taylor (1967) argued for a simpler "multiple talent 
approach" where teachars are helped to identify and nurture students' multiple 
talents in five talent areas: productive thinking, decision making, planning, 
forecasting, and communication (Table 4, Goal A). Thus, it is argued that 
traditional academic talent helps students gain knowledge in academic disci- 
plines while the talent areas (thinking skills clusters) help students process 

33 



Er|c ^2 



or use the knowledge to create new solutions to problems (Schlichter. 1986). 
Because a major goal of the program is to integrate knowledge and talent, 
teachers are trained to provide systematic opportunitias for students to de- 
velop skill clusters in all five talent areas and apply them to academic learn- 
ing as well. Thus, in addition to training .eachers to recognize talent areas 
.in children, the program developers produced a series of binders filled with 
suggested talent activities. 

Each activity is structured to include ideas for classroom motivation, sug- 
gested teacher directions and discussion, and sample student responses. More- 
over, teachers are encouraged to use the talent areas in academic learning 
situations. For example, in the talent area of productive thinking (to gener- 
ate multiple, varied, and unusual ideas or solutions and add detail to ideas to 
improve or make them more interesting), a sample activity for a math unit on 
surveying and graphing directs students to think of a variety of unusual topics 
for a survey they will conduct and graph during the day. Teachers are taught 
that an important aspect of the activities is open discussion with students 
about what is happening as they work through a problem, to help students de- 
velop awareness and understanding of strategies and their use in each talent 
area (Table 4. Goal B) , 

In addition, by providing opportunities for students to use the talent pro- 
cesses in each subject area, teachers are to help students develop understand- 
ing of ways in which the processes will help them further develop their knowl- 
edge and understanding (Table 4. Goal C) . Finally, it is expected that stu- 
dents' self -concept and confidence will be increased as they recognize and 
learn to use the five talent areas, and see chat academic talent is but one of 
many talent areas needed for further learning (Table 4. Goal D) . 
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Table 2 provides a sununary of additional program characteristics. TU is 
an inservice model designed to help teachers develop knowledge, skills and the 
disposition to nurture the five talent areas in children. Instead of providing 
paper-and-pencil exercises, program developers provide extensive training for 
teachers so they are able to play a critical role in mediating students' devel- 
opment in the talent areas. Since the processes for each talent area are used 
across the various subject areas (as in program exercises in Odyssey), and stu- 
dents are encouraged to discuss and reflect on the use of the processes, this 
program encourages teachers to teach directly for transfer. Training sessions 
focus on four areas: (a) teaching classroom teachers about mtiltiple talent 
theory and talent skills definitions, (b) modeling and demonstration of teach- 
in;; skills specific to each cluster of thinking skills in the multiple talent 
model, (c) classroom practice sessions during which teachers receive structured 
feedback on their performance, and (d) one-to-one and small-group planning 
sessions in which teachers grapple with ways to implement the program in their 
particular classroom setting (Schlichter, 1986^ . 

Evaluation efforts have largely focused on the success of trhe model as an 
inservice model for teachers, with some focus on evaluation of studenr. perfor- 
mance. Specifically, program developers have focused on examining the extent 
and nature of program implementation among teachers receiving the training. Ex- 
tensive analysis of workshop ratings by teachers indicate that the staff devel- 
opment component of the program is effective in enhancing teachers' understand- 
ings, skills and attitudes related to implementing the model (Crump, Schlichter 
& Palk, 1988; Schlichter, 1986). In addition, program developers have examined 
the impact of the multiple talents model on higher order thinking skills 
through the use of pre- and posttest measures, including the Torrance Tests of 
Creative Thinking, the Coopersmith Self -Esteem Inventory, the Stanford 
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Achievement Test, and the Criterion Reference Tests (a battery of 10 measures 
developed b" project staff and a university research team designed to assess 
changes in talent areas) . 

Four experimental and control schools were included in a three-year 
study. Mixed results from the standardized tests indicate some improvements in 
some grades during some years. Results from the internally developed criter- 
ion- referenced tests indicate success at some grade levels as well (Schlichter, 
1981). Thus, there appear to be some gains in experimental groups in overall 
academic achievement and talent areas over control groups, although results are 
not conclusive. 

Philosophy for Children 

The Philo.«; j for Children Program was developed by Matthew Lipman for 
children in grades K through 10, with the future goal of developing materials 
through 12th grade. The broad goal of the program is to help children "dis- 
cover their intellectual capabilities" (Lipman, 1985). An underlying assump- 
tion is that the act of engaging in philosophical inquiry about various issues 
develops thinking skills. That is, by thinking about thinking, it is assumed 
that children will learn skills such as making connections, drawing distinc- 
tions, defining and classifying, and assessing factual information (Table 4, 
Goal A). Moreover, repeated practice of these skills in contexts that interest 
students ^.s intended to facilitate tho development, in students, of "cognitive 
dispositions" showing a rear»^ness to employ the skills they learn (Table 4, 
Goal D; also see Lipman, 1985). Philosophy for Children is used in over 4000 
American elementary and secondary schools, and parts of the program have been 
translated into French, Chinese, Hebrew, Spanish, German, Portuguese, Danish, 
and Arabic (Chance, 1986). It is most appropriately used with average and 
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above-average students, since below- average students may have trouble reading 
the program novels (unless a novel intended for younger children is used) . 

Students read novels that were written to be nodels of inquiry and to il- 
lustrate for students basic principles of formal and informal logic. In the 
six novels, young characters apply philosophical thinking to everyday experi- 
ences. Three novels are provided for use in the elementary grades, pixie is a 
novel for children m grades through four and involves reasoning about lan- 
guage. Kto and Gus is also intended for use with grades kindergarten through 
four and centers around reasoning about nature. Harrv Stottlemeier' s Discov- 
ecr, the first to be written, is used in grades four through seven and focuses 
on developing basic reasoning skills. The other three novels, intended for use 
in grades 7 through 12, include reasoning about ethics, reasoning in language 
arts, and reasoning in social studies. Thus, the focus in the early grades is 
on more general reasoning skills and experiences. As the children in the pro- 
gram get older, the ties to subject matter become more explicit. 

Table 2 summarizes additional features of the program. Exercises accompa- 
nying the novels afford students the opportunity to practice and assess par- 
ticular thinking skills such as syllogistic reasoning, formal analogy problems, 
informal reasoning, and creativity and verbal fluency. The objects of thought 
in the discussions of the novels are philosophical issues. During the exer- 
cises, the focus is on the reasoning skills to be developed. These activities 
are intended to take place within a "community of inquiry," where students and 
teachers engage in dialogue focused on evaltiating the content and quality of 
the dialogue (Bransford et al., 1985), 

Insterid of focusing on teaching the skills separately thrrugh the exer- 
cises provided in the program, Lf^pman (1985) argues for the need to help stu- 
dents coordinate thinking skills, which requires modeling from the teacher 
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during philosophical discussions, as well as encouraging dispositions to employ 
the skills in a coordinated fashion (Table 4, Goals B, C, D). As with other 
programs with multiple goals, the teacher's role is to encourage and support 
students as they reflect on their discussions and to help them become aware of 
their thinking processes through the discussions of the novels and participa- 
tion in exercises. The hope is to facilitate and reinforce an interest in 
philosophical questions that will 'ast beyond the duration of the course so 
that students will show improvement in their ability to think and in what they 
think about (Nickerson et al., 1985). The program is intended to be used for 
three 40 -minute periods per week, which is a substantial time investment in an 
elementajry or secondajry curriculum. 

Philosophy for Children requires extensive teacher training. Lipman 
(1985) maintains that guiding philosophical inquiry is a skill to be learned, 
and it is best learned in the same fashion children do. Therefore, he recom- 
mends that teachers participate in reading, exercises, and discussions during a 
two-week summer workshop with experienced trainers. One-year training is also 
available where trainers visit the teacher's school weekly to provide training 
and supervise the teacher's implementation of the program. Unlike SOI, Odys- 
sey, ThinkAbout, and SAPA, the developers assume teachers need more than commit- 
ment to teach thinking skills and more guidance than use of teachers' manuals. 
Like IE and TU, this program requires considerable time investment in learning 
to use the program properly, and considerable skill in implementing it. 

Sternberg and Bhana (1986) report there is little evidence available about 
use of any of the novels except. Harrv Stot tlemeir's nisrovpry with childre.i in 
grades four through seven. Instead of focusing the evaluations on component 
parts of the program, it has been evaluated as a whole. Therefore, little in- 
formation is provided on subject drop-out, class selection, durability, 
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transfer, subject population, experimenter bias, and statistical analysis. 
They also raise the issue of interpretive problems since most evaluation 
studies are reported in Lipman's own journal. Thinking . Evaluation data show 
that different groups of students appear to make gains in different areas, but 
there is no explanation as to why (Bransford et al. , 1985). The greatest gains 
seem to be in verbal tests of critical thinking abilities, which makes sense in 
light of the heavy use of reading and discussion (Sternberg & Bhana, 1986). 

Nevertheless, despite the lack of specific experimental evidence to sup- 
port the program's effectiveness, there do seem to be some promising aspects. 
Transfer is more likely because of the use of everyday situations in the novels 
and because of repeated opportunities to reinforce earlier learnings across the 
grades. Informal comments on the program reveal a positive attitude toward it 
from teachers and students. Reports of enthusiastic discussions where students 
actively participate in philosophical inquiry and carefully listen to one 
another is important evidence for the likelihood of developing long lasting 
interests in philosophical inquiry. However, there is no information available 
regarding the general question of the cost of using the program (e.g., time 
invested, training requirements, problems of continueu use across 
the grade levels) compared to the benefits realized. 

Programs That Use Lan^ape and Sy m bol Manip t.' ''ti. n 
Reciprocal Teaching, the Expository Writing Program, nnd LOGO were not 
initially designed to teach thinking skills. Instead, th^^\ we-' designed to 
focus on improving knowledge, understanding, and use of three sKill areas: 
reading comprehension, written composition, and computer programming. They are 
included in this review because program developers claim that developing these 
skills is not just an end in itself but also enables students to explore and 
understand subject matter concepts and related skills. It is as£.umed that 
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reading, writing, and programning are occasions for students to learn more 
about the content areas typically studied in school, ways to solve problems, 
and to develop a better understanding of and skill in using occasions to learn. 
This, in turn, will improve one's thinking about subject matter content. The 
methods used in each program include highly specific teacher support for learn- 
ing, where a mediated learning environment is seen as key to the program's suc- 
cess. Materials in each program are specifically related to the skill areas 
imder development. 

Reciprocal Teaching 

Reciprocal teaching started as a set of procedures designed to help upper 
elementary students develop reading comprehension skills. Its originators. 
Amunarie Palincsar and Ann Brown, wanted to develoo a way for students to de- 
velop and practice the deliberate use of understanding processes when they read 
text material (sae Table 4, Goals A, B, & C; also, see Reeve, Palincsar, & 
'.rown 1987). An underlying assumption of the instructional procedure is that 
if student." learn to use comprehension strategies deliberately and become aware 
of ways to control their use strategically, they will learn to think critically 
about material they reaa in academic subjects. 

Instead of skirting the issue of prior knowledge in thinking, this program 
rests on the assumption that the ability to comprehend new information is an im- 
portant part of thinking and problem solving. The overal ,,oal of using this 
instructional procedure is to h-lp students learn to use and value "deliberate 
effortful cognition" to address problems requiring academic thinking skills 
(Table 4, Goals B & D; also, see Reeve et al., 1987). A secondary goal for the 
program is to help students develop knowledge about thinking so that they 
understand how the methods they are learning in reading comprehension influence 
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the thinking process (Table 4, Goal C) . Reciprocal teaching began as a series 
of training studies, and since then its use in classrooms has increased. 

The program centers around teaching students to use four key strategies as 
they read regular text materials in the classroom: summarizing, questioning, 
clarifying, and predicting. These strategies are considered key because they 
can be used to foster comprehension skills and comprehension monitoring activ- 
ity (Reeve at al., 1987). Instead of just learning to do a thinking activity, 
students are also taught to assess the quality of their activity. 

Table 2 includes a sximmary of additional program characteristics. The 
role of the teacher in fostering understanding and use of the four key strat- 
egies is critical. The main teaching activity centers around carefully con- 
structed dialogue about text passages. Teachers must first ascertain where in- 
struction should begin. This might entail beginning with discussions that cen- 
ter around what a oiimmarizing or questioning activity is, and how it is used. 
Time is also spent providing a rationale for learning the comprehension strat- 
egies so that students are motivated to learn the strategies and see how they 
apply to learning in the subject areas. This is followed by careful modeling 
of appropriate use of the strategy by the teacher, including sharing her 
thought processes :kbout the strategy use. 

As students begin using the strategies, the teacher diagnoses difficulties 
on the spot, gives feedback to students, and gradually expects them to demon- 
strate more competence in strategy use. Gradually, the teacher asks the stu- 
dents to take on the role of teacher and begin using the strategies with the 
class or a group, thinking aloud about the process, receiving feedback from 
teacher and students, and gradually increasing competence in strategy use. The 
nature of this process is gradual because of the underlying assumption that 
teaching this process requires anticipation of students' competence, so that 
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they are continually at the edge of their learning capacity. Such knowledge re- 
quires the teacher to carefully monitor the students' level of competence and 
continually adjust the amount and type of support provided (Reeve at al.. 
1987). 

Taking on this role requires extensive '^raining for teachers in several ar- 
eas. First, teachers must understand the appropriate use of the four strat- 
egies (summarizing, questioning, clarifying, and predicting), and how they ap- 
ply to increased understanding of a particular passage (and ultimately to the 
content being studied). Without a thorough understanding of the text material, 
teachers could miss opportunities to show students the value of such activities 
as clarifying and predicting, and such activities could be reduced to trivial 
exercises. 

Second, teachers need to learn to model the use of the strategies. It is 
one thing to be a good reader and another to tell others how one goes about com- 
prehending the material (Pearson & Dole, 1987). Third, teachers need to learn 
how to -scaffold- (Palincsar & Brown, 1985) so that the teaching process 
becomes one of gradually relinquishing to the student responsibility for using 
the process. Students need the opportunity to practice the strategies with 
support, but that support must eventually be removed at the appropriate time so 
students can learn to apply them independently. In addition, this gives 
teachers the opportunity to see how well students can use the strategies and 
know when remediation is necessary. 

Palincsar and Brown have provided training for teacher.-: in the above ar- 
eas, but the extent to which they address the issue of how the four strategies 
specifically or uniquely apply to different subject areas is unclear. They do 
give sample dialogues th&t apply to specific text material (Palincsar, in 
press; Palincsar & Brown, 1985), but it is not apparent that the nature of the 
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content is directly related to the nature of the strategies. For example, what 
does -questioning- or -clarifying- look like in science and social studies? 
Are the activities different for each? What is the nature of the issues that 
arise in each area? Issues such as these might be more specifically explored 
and dealt with directly in teacher training, especially since Reeve, Palincsar, 
and Brown (1987) advocate use of these strategies in other subject areas. 

Use of Reciprocal Teaching has been extensively evaluated, and these ef- 
forts provide inspiration that thorough evaluation of complex educational inter- 
ventions is possible (Pressley et al., 1985). Pearson and Dole (1987) summa- 
rize outcomes of several sets of studies. One set of studies evaluated the use 
of the reciprocal teaching procedure to measure poor readers' daily independent 
comprehension test scores immediately after the intervention, eight weeks 
later, and six months later. In all instances scores increased from 40X to 75% 
correct. When standardized measures of comprehension, social studies, and 
science were compared to a control group's measures in these areas, those 
participant's gains were consistently greater. In addition, a series of trans- 
fer tests showed reliable improvement in reading and writing. Tb^ use of the 
procedures was also compared across different types of teachers (master teach- 
ers, average teachers, and peer tutors), and all three groups achieved suc- 
cess. In a series of comparative studies, reciprocal teaching showed sig- 
nificant gains over practice-alone treatment or isolated skills treatment. Fi- 
nally, teaching methods were compared and It was found that the reciprocal 
teaching method was superior to teaching where students did not gradually as- 
sume responsibility for their own learning, and where students assumed responsi- 
bility according to a predetermined schedule. 

These findings Indicate a high degree of success (Including stability of 
treatment over time and transfer to reading and writing activities), but also 
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indicate that the progran's success is due to using its methods correctly as 
well as focusing on the key strategies. Apparently the method (carefully con- 
structed dialogue and appropriate scaffolding) is just as important as the pro- 
gram content. In this program, the role of teacher as mediator is as key as 
it is in IE, Odyssey, tU, and Philosophy for Chil''>-en. 

Expository Writing Pr»gr"ni 

The Expository Writing Program was developed for use with upper elementary 
and early middle school students, and grew out of a three-year study designed 
to improve students' comprehension and composition of expository text 
(Kirschner, Raphael, & Englert, 1986; Raphael, Englert & Kirschc-jr, 1986; 
Raphael & Kirschner, 1985; Raphael, Kirschner & Englert, 19S6a, 1986b). The 
"program- consists of a manual for teachers who wish to implement a process ap- 
proach to writing as it was designed and implemented in the three-year Teaching 
Expository Reading and Writing Project. Like Reciprocal Teaching, it started 
with a focus on developing school related skills (in this case, reading and 
writing). Because of its emphasis on teaching students strategies that involve 
problem-solving and delineating emergent problems through reading and writing, 
it is included in this review of programs that teach thinking skills. 

Assumptions that underlie the program come from research in three areas: 
question-answer relationships, expository text structures, and the writing pro- 
cess. The goals of the EWF are to help students (a) incegrate information from 
across texts; (b) leani aoout and use appropriate text structures to comprehend 
written material and create their own written text; and Cc) learn about and use 
the writing process to find information, plan, edit, and revise different types 
of texts (Table 4, Goals A, B, C) . 

Thus, the developers see writing expository text as two distinct, but in- 
terrelated problems. One problem is to help students understand underlying 
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structure of text material so they can better comprehend it (and eventually 
write about it) . In this way, their goal is similar to that of Reciprocal 
Teaching, where students are taught ways to better comprehend text material 
(Table 4, Goal A). These two programs also share a focus on helping students 
become aware of their own learning process so they can use it to monitor their 
own learning and become independent readers and writers (Goal B) . 

A second problem goes beyond the goals of Reciprocal Teaching to help stu- 
dents learn to manage writing about subject matter content as well. This pro- 
gram focuses on teaching students to understand the underlying structure of 
social studies texts and to use different text structures in thsir own writing 
(Goal C). In addition, research on the writing process influenced the develop- 
ers to focus on showing teachers how to develop an appropriate environment for 
writers so that the non-linear writing process (find information, plan, edit, 
revise) would be used as a tool for their thinking and writing, and not as a 
set of assignments to fulfill (Raphael & Kirschner, 1985). 

Additional features of the program are summarized in Table 2. The manual 
(Raphael, Kirschner, & Englert, 1986b) consists of a rationale for the program, 
a set of directions for teachers to follow, and sample materials to use with 
students to teach them about text structures and to teach them to use the plan- 
ning, editing, and revising stages of writing different types of text (e.g., 
comparison/contrast, problem/cause, problem/solution, explanation, story). 

Like Recip>-ocal Teaching, this program emphasizes the importance of the 
teacher's role in the learning process. Teachers are encouraged to conduct 
discussions carefully that make explicit their own and their students' thinking 
processes as they read text and plan their writing. Thinking about comprehen- 
sion and writing tasks is explicitly modelled by the teacher, not described or 
assumed. Like Reciprocal Teaching, it is also assumed that teachers will 
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continually monitor students' current level of understanding and skill and 
attempt to provide appropriate scaffolding along t..e way. Peer-editing groups 
guided by the teacher are also a means by which students practice skills and 
become more aware of thinking processes during the writing process. The pro- 
gram developers claim that the strength of the program lies in the interconnec- 
tions made between the reading and writing process, where students are shown 
how to bring both processes together to comprehend and write about social stud- 
ies content. 

During the thr je-year study, researchers worked collaboratively with class- 
room teachers to develop program materials. It appears that the manual 
(Raphael et al.. 1986b) is intended as a sufficient resource for teachers to 
use on their own if they wish to implement EWP in their classrooms, since there 
is no discussion in the manual of further training needed, while the manual 
clearly states what the teacher should do to implement the plan, it is uncer- 
tain that any teacher automatically has the knowledge or skill needed to pro- 
vide such closely guided instruction. 

Formal study of the program has focused on changes in teachers' knowledge 
and implementation of the writing program (Kirschner et al.. 1986) and their 
knowledge of the writing process (Raphael et al.. 1986a). It was found that 
after participation in the EWP. teachers viewed writing ai a process where 
genuine purpose and audience are essential, and saw writing as a form of commu- 
nication instead of a place to assess skills. Information about student perfor- 
mance is limited to examples showing improved ability to organize text and con- 
vey information, improved ability to write narratives, and improved attitudes 
toward writing. No studies have been conducted to assess h-w students might 
transfer what they learn about text structure and wricing in the social studies 
to writing about other content areas, writing about social studies content in 
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other contexts, or their general ability to analyze text material for its 
underlying structure. 

LOGO 

LOGO is a computer language suited for use by children throughout the ele 
mentary years. Its use has been popularized by Seymour Papert (1980). Becaus 
it requires the availability of microcomputers in classroons, its use has as 
yet been limited; however, computer use in classrooms is increasing all rhe 
time (Nickerson et al., 1985). Progranuning in the LOGO language provides chil 
dren the opportunity to explore mathematical and logical concepts. The com- 
puter is used as a tool to learn about ^ireas such as: problem- solving strat- 
egies, fractions, algebra, word problems, trigonometry, and geometry. 

In additio.i to helping students learn mathematical content, program devel 
opers claim that students will also develop problem- solving skills through pro 
gramming in LOGO (Table 4, Goal A). Thus, it shares with Reciprocal Teaching 
and EUP the goal of promoting generalized skills (e.g., problem-solving) 
through developing and using another skill, in this case, programming. This 
review focuses on the- cla'm that use of LOGO improves general problem solving 
abilities and looks at recent changes in program goals and learning environ- 
ments in which LOGO is used that might improve the likelihood of transfer of 
problem solving strategies to other doirains. 

Papert 's initial recommendations for using LOGO with children minimized 
the role of the teacher. He claimed that teachers need to learn the LOGO lan- 
guage so they can teach it to students, but he recommended letting students ex- 
plore and discover uses on iheir own. However, a study that investigated the 
extent to which students transfer planning skills from the activity of program- 
ming to other experimental tasks that also require planning (but did not 
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involve computers) found no effects (Pea & Kurland, 1984). Instead, the chil- 
dren "played" with LOGO, but ignored the conceptually challenging aspects of 
LOGO to which Papert claimed they would gravitate. The teachers involved in 
the study concluded that they bad to plan more specifically what they intended 
for students to get out o„ using the language, and they needed to provide appro 
priate support for helping them realize those goals (Newman, 1985) . Unlike Re- 
ciprocal Teaching and EWP, these LOGO user^ left out the careful teacher goal- 
setting, modeling, and support that is needed to help students develop an aware 
ness and appreciation of the value of processes such as planning in improving 
thinking. Thus, in the way LOGO was originally implemented in classrooms, It 
focused on the single goal of developing problem- solving skills, and used a non- 
directive approach to instruction. 

More racent attempts to use LOGO to develop generalized problem-solving 
skills are moving in a different direction. For example. Black and his col- 
leagues (Black et al., 1988) describe efforts to re- think and reorganize the 
pedagogy .-.urrounding LOGO use. Thesa researchers reviewed several studies on 
the effectiveness in using LOGO as a tool to teach problem solving (Swan & 
Black, undated) and stvdied their own use of LOGO with a group of children in 
the fourth through eighth grades. They argue that positive effects resulted 
from LOGO use that used a particular kind of pedagogy: (a) teachers and stu- 
dents focused on specific aspects of the general problem- solving process (e.g., 
subgoals formation, forward chaining, backward chaining, systematic trial and 
error, alternative problem representation, analogical reasoning); (b) teachers 
provided direct instruction in the identified component skills; and (c) a high 
degree of teacher mediation was required in the learning environment. Other re- 
searchers (e.g., Emihovich & Miller, 1988; Lehrer, 1986) also encourage taking 
a close look at the environment in which LOGO is used and the nature of the 
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jjocial as well as cognitive development that takas place over time to under- 
stand why and how the use of LOGO helps students generalize the skills they are 
developing to further learning situations. 

laese recent changes in the pedagogy surrounding the use of LOGO (identifi- 
cation of specific component skills that are directly taught [Table 4, Goal A], 
a high degree of teacher mediation, more attention to ways in which teachers 
must scaffold students' learning across experiences [Goal B]) show a movement 
toward creating features of a learning environment that encourage transfer (see 
Table 2). Indeed, results from pre- and posttests given to a group of fourth 
through eighth graders (Black, Swan, & Schwartz., 1988; Swan & Black, undated) 
indicate improvement in strategies, and transfer of these strategies to 
noncomputing domains. More studies of this nature are required to leum more 
about the potential for tr-vnsfer of problem solving strategies learned in a 
LOGO-based problem-solving program to noncomputing situations. 

Summary 

The eight programs reviewed in this section share a focus on multiple 
goals designed to improve thinking. Each program claims some success at foster- 
ing better thinking, but only three (Reciprocal Teaching, Odyssey, and LOGO) 
provide specific evidence of transfer of such abilities to other domains (Table 
2). However, ..ike the first group of prograros reviewed, these programs do re- 
veal some quf^lities that are important to consid<?.r in working toward improving 
thinking skills. Table 2 serves as a useful summary of program characteristics 
to be highlighted in the discussion that follows. 

Multiple Goals Are Ne< >dt5d 

What qualities in programs enhance the likelihood of transfer? This is a 
question of how to get students to transfer use of skills and strategies (Table 
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2, Goals A & 3) they have learned in one situation to having the conditional 
knowledge (Goal C) , awareness and disposition (Goal D) to use it in others. In 
other words, it is a question of whether students have access to the intellec- 
tual skills they have developed (Prawat. 1988). Of the ^ight programs reviewed 
in this section, four cake n and give specific attention to all four goals: 
Odyssey. TU. Philosophy for Children, and Reciprocal Teaching. Reciprocal 
Teaching provides specific evidence of transfer of skills to other domains. 

Test results on a small student sample ot seventh-grade students using Od- 
yssey show improvements in qualitative aspects of thinking (v?.g.. appropriate- 
ness of design, clarity of expression, and use of supporting reasons), so some 
evidence of students being able to use -he skills in educationally and practi- 
cally relevant tasks is provided. Study of overall achievement did not show a 
pattern of improvement as a result of TU teaching, yet teachers report much stu- 
dent e^.thusiam and widespread use of the talent areas in their classrooms 
(Barbieri. 1988; Schlichter. Hobbs. & Crump. 1988). While there is no specific 
evidence from formal studies of transfer of skills taught in Philosophy for 
Children to other situations, there are abundant informal reports that enthusi- 
astic discussions, positive attitudes toward philosophical discussions, the use 
of everyday situations as focal points for ap- /.ng philosophical thinking, and 
repeated opportunities to develop an<: improve skills across the grades all 
point toward the likelihood of transfer. 

Two programs, IE and the Productive Thinking Program, take on three of the 
four goals, so that they attempt to he2p students learn skills, learn strat- 
egies for using them, and aisc attempt to develop positive attitudes and dispo- 
sitions to use the skills. Like Odyssey, TU, Philosophy for Children, and Re- 
ciprocal Teaching, these two program.c intend to help students experience and 
learn r.o enjoy the thrill of discovery, and the satisifaction gained in pursuing 
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one^s thinking. If students develop an awareness of their own thinking strat- 
egies (Table 2, Goal B) and a positive attitude toward the work it takes to 
think more clearly (Table 2, Goal D) , it is more likely they will be motivated 
to use the skills in other situations. Evaluation efforts show internal sue- 
cess in both programs, so that students do show gains in target skills when 
similar test iteas are used. More study of each progrrja is needed to document 
whether transfer does occur when students are not also taught more specifically 
when to use particular knowledge and skills (Table 2, Goal C) . 

EVP works towa?:d three of the four goals (Table 2, Goals A, B, C) , leaving 
development of attitudes and dispositions (Goal D) as a potential outcome that 
is not directly addressed. Since little study of student outcomes has t.xken 
place, it is difficult to address the issue of transfer of the use of skills de- 
veloped to other domains. Further study is needed. 

Finally, people using LOGO are changing and adding goals as they learn 
more about its potential success. Current use focuses on developing two goals 
(Table 2, Goals A & B) and the added focus on developing strategic knowledge 
and skill seems to make a difference in its success. Students need help in be- 
coming aware of the generalizability of the strategies they are using. More 
studies need to be done to determine whether focusing on two of the four goals 
is adequate for successful transfer of problem-solving strategies to noncom- 
puter domains. 

Thus, specific instruction and support in working toward multiple program 
goals point toward increasing the likelihood of transfer. Working toward Goal 
A is essential; skills must be taught. However, access to such skills is also 
essential if students are to be able to use them in further learning situations 
(Prawat, 1988). Therefore, also working toward Goals B, C. and D will more 
li^v'ely ensure that students will use cognitive processes widely and frequently 
(Resnick, 1987). 
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Teaching Mt^t-hnrf ^ and Material.? 

Six of nhe eight programs use content-related materials (Odyssey, TU, Phi- 
losophy for Children. Reciprocal Teaching. EWP. LOGO), while two use general ma 
terials that take the focus off subject matter content (IE. Productive Thinking 
Program, [see Table 2]). GJyssey purposely uses "content- rich" materials to in 
crease the likelihood of transfer. TU inservice trai Ing encourages teachers 
to find and use opportunities to develop talexit areas in all subject area learn, 
ing. Philosophy for Children uses a particular genre of children's literature 
to promote and provide a context for philcsophical inquiry (Lipman. 1987). Re- 
ciprocal Teaching and EWP use school subjects and materials as the focus of 
their instruction. LOGO uses a progr/maning language that embodies a specific 
type of problem solving that draws on conceptual schemes related to computers 
algorithmic thinking, procedural thinking, logical debugging, modularization 
(Nix. 1988). In these cases, the connections the program experiences have with 
content seem to challenge students and promote interest. Moreover, it is 
assumed that the subject matter will provide stimulation to think critically. 

Although the Productive T ..nking Program limits its exercises to everyday 
situations, it uses a comic format and everyday characters and situations in- 
tended to interest students and hold their attention. IE developers report 
that student5< find the generic exercises "intrinsically interesting" as one 
would find a puzzle fascinating to solve. However, since the focus in this pro- 
gram and in the others is on the lively interaction between and among teachers 
and students (in contrast to programs like SOI and CoRT where interactions are 
rather routine) . there seems to be a complex interaction between the kinds of 
materials used, the nature of interaction surrounding the materials, and the 
likelihood of transfer. Also, it should be noted that those studies that 
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•-eported the greatest internal success iu implementation of IE were ones where 
IE was taught in conjunction with other subject matter of interest and 
importance to the students (Savell et al., 1986). Thus, while contenc-relaced 
materials seem to be more interesting to students, mere interest is not enough. 

All programs reviewed in this section call for a high degree of teacher me 
diation. It seems that a particular kind of learning environment is required 
if skills and strategies stu<Jents learn will be valued and used in further 
iearninj situations. Moreover, without specific focus on how and when co use 
the skills (Table 2, Goal C) , there may be less likelihood that students will 
know how and when to use the skills in other situations. Lively dialogue, col- 
laborative exploration of content and strategies, and careful scaffolding (of 
skill development, strategy awareness and use, development of conditional knowl 
edge, and development of the disposition to use the skills) increase the likeli 
hood of success in promoting transfer of skills to further learning situations. 

Teacher Training 

Five of the eight programs (IE, Odyssey, TU, i^iilosophy for Children, Re- 
ciprocal Teaching) provide and advocate extensive teacher training (Table 2). 
For example, studies reporting internal success for IE implementation note that 
instructors using the program had at least a week of training (Savell et al., 
1986). The Productive Thinking Program and EWP assume careful and motivated 
use of the materials is adequate (although EWP did begin with careful work with 
teachers initially). Current LOGO users seem to be heading in the direction of 
helping teachers think more clearly and concisely about the nature of tne envi- 
ronment in which problem solving is successfully learned and generalized. When 
programs with multiple goal :re offered, it does seem that teacher training 
should be offered. For example, as Reciprocal Teaching is described in the lit- 
erature (Palincsar, in press; Palincsar'6e Brown, 1985; Reeve at al., 1987), ic 
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is an extremely complex activity, ic r<.quires rich knowledge of the key strat- 
egies being taught, techniques for determining student progress, ways co struc- 
ture dialogues to provide appropriate support, and content in the written 
text. It seems unrealistic to assvune that any program that promotes thinking 
can simply be carried out as a "how to" by any teacher. 

The Persis tent Question of Tran.«;f o>- 
There is little definitive evaluation evidence that programs promoting gen- 
eral thinking skills are effective in helping students think better in general, 
and yet the available evidence points toward characteristics of some of the pro- 
grams that seem promising. In this final section, a summary of important pro- 
gram characteristics is provided. This is followed by a discussion of issues 
and difficulties in evaluating the success of thinking skills programs. 

Program Development 

rne overall goals of the programs shape their content and methods, so pro- 
gram goe.ls are an extremely important element in the development of any effort 
to teach students- higher order thinking skills, m general, the programs re- 
viewed focus on teaching students to use -analytic and reasoning skills in 
problem-solving situations in academic and everyday contexts and try to communi- 
cate general principles by which students will learn to apply their skills. 
Some programs use subjec: matter-oriented materials as well, but their focus is 
still on developing skills and strategies more than on developing understanding 
of content. The first group of programs reviewed center their efforts on the 
single goal of developing particular thinking skills, and the second group of 
programs focus on helping students work toward multiple goals (see Table 1). 

This review has shown that studer can benefit, at least in iirinediate 
gains iP specific contexts, from purposeful teaching and prsotice of specific 
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skills (Table 1, Goal A). In addition, three other outcomes (developing meta- 
cognitivf awareness and strategies [Goal B] , developing conditional knowledge 
(Goal C], developing appropriate attitudes and dispositions (Goal D]) are also 
important for several of the programs. Nickerson et al. (1985) argue that all 
four outcomes are necessary for successful teaching of thinking. Why are all 
four necessary? 

Successfully teaching of thinking involves getting students to transfer or 
habitually use abilities or skills in situations beyond program exercises. The 
overarching goal is to do more than teach higher order thinking during program 
exercises; higher order thinking should take place in everyday and learning ex- 
periences. Most of the programs claim transfer will occur by participation in 
the program, but are unable to provide evidence to support that claim. What 
seems to be lacking in many of these programs a^e direct efforts at promoting 
transfer in program materials and methods; without such efforts, transfer does 
not just happen. Thus, instead of placing transfer as an implicit or under- 
stood outcome of the program, perhaps it should be an explicit outcome that 
holds equal importance with the other four outcomes for programs. Programs 
that explicitly teach for transfer (see Table 2) do show promise. 

If transfer is an important outcome for a yiogram designed to teach higher 
order thinking, what are important ingredients for program developers to keep 
in mind? Poison and Jeffries (1985) offer four criteria for analyzing whether 
programs are consistent with current research on problem solving: developing ex- 
plicit cognitive objectives that reflect an explicit model of thinking and prob- 
lem solving; using a wide variety of explicit problem- solving techniques; devel- 
oping a control schema or management strategies; and providing uset \l examples 
and problems of graded difficulty that provide drill and practice. When these 
four criteria are followed, it seems that programs can succeed at showing 
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immediate gains in using program exercises oi other similar materials (e.g., 
CoRT, Productive Thinking Program), but they do not necessarily show transfer 
gains . 

Sternberg (1987) offers some additional guidelines intended to increase 
the likelihood of transfer in a thinking skills program. These guidelines are 
com.itent with program characteristics described in this review regarding 
transfer (see Table 2). First, he recommends specific training for developing 
executive skills (Table 2. Goal B) . Students need to be taught to be aware of 
and manage their use of the target skills (e.g.. Reciprocal Teaching. Phi- 
losophy for Children. TU) . Second, general principles or rules for thinking 
must be taught in the context of a variety of disciplines (Table 2: Materl- 
als). This allows students to understand how the principles cut across subject 
matter areas. Odyssey. ThinkAbout. TU. and Reciprocal Teaching provide materi- 
als and experiences across disciplines, but only Reciprocal Teaching explicitly 
directs, teachers to discuss how and when the skills can be used in various 
subject matter domains. Also, the question was raised earlier as to whether 
Reciprocal Teaching considers differences in how the key strategies apply to 
various subject areas (e.g.. Is summarizing, questioning, clarifying, or 
predicting different for science and math, and if so. how?). Thus it is also 
important to raise the issue of the limitations of the generalities being 
taught and to examine closely when particular thinking skills might best be 
taught ia specific subject matter contexts (Alexander & Judy, in press; 
Resnick. 1987). 

A third recommendation Sternberg (1987) offers is that priijciples should 
be presented in contexts that range from the abstract (without regard to par- 
ticular content) to concrete (particular instances; see Table 2. Goals B & C) . 
Philosophy for Children's emphasis on the value of "thinking about thinking" in 
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addition to its emphasis on applying logic to specific situations is a good ex- 
ample of this blend of experiences. Fourth, Sternberg recommends that contexts 
in which principles and rules are presented should vary from the academic to 
the practical so students can see instances of how principles that apply to 
their academic learning also apply to everyday situations (Table 2, Goal C) . 
Odyssey, TU, and ThinkAbout probably come the closest to providing a mixture of 
both kinds of contexts. However, they do not explicitly show students what the 
relationship is between applying principles in both instances, and the mere 
presence of both kinds does not guarantee that students will see and make the 
connections on their own. TU seems to rely on teachers' knowledge and under- 
standing of such connections in order to make them explicit to students. 

Fifth, Sternberg (1987) argues for using multiple media of instruction 
(e.g., lecture, discussion, reading, writing, individual and group projects) to 
ensure the likelihood that students will internalize what is taught in the pro- 
gram (Table 2: Intended Teacher Mediation, and Materials). This recommendation 
seems to reflect good pedagogy for helping students understand principles and 
practice skills and enjoy the learning process; however, without explicit atten- 
tion to how they are reflected in various experiences (see Table 2, Transfer of 
Skills), there is not necessarily more likelihood that internalization will oc- 
cur. 

Finally, Sternberg argues that while individualization is difficult to 
achieve, some measure of it is necessary to account for individual differ- 
ences. Some programs, such as SOI, are extremely individualized with indi- 
vidual prescriptions for students based on a diagnostic test. However, SOX 
evaluation results do not show evidence of transfer to nontest-like materials. 
Reciprocal Teaching is also diagnostic and somewhat individualized, but in a 
more flexible way, where the diagnosis occurs throughout instruction and 
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further instruction is delicately adjusted according to the teacher's percep- 
tion of what students need." Careful scaffolding proves more successful in 
leading to transfer than individualization that is done in a pre- and posttest 
fashion (Table 2: Intended Degree of Teacher Mediation, Teacher T.-aining) . 

If transfer is taken seriously as an important and explicit outcome of ef- 
forts to teach higher order thinking, the above recommendations should be taken 
as a set, rather than selectively implemented (Table 2, Goals A, B, C, D) . 
Transfer requires explicit focus of instruction on metacognitive awareness and 
strategies in addition to teaching specific abilities or skills. In addition, 
it requires showing students how and why principles and skills apply in a vari- 
ety of contexts (various academic subject areas; academic and practical; ab- 
stract and concrete). These insights do not happen on their own. Finally, it 
v<-<quires good pedagogy (use of multiple media, and suited to individut^ls a^j 
well as to the group) and appropriate scaffolding. In the following section, 
recommendations are given for specific methods and activities that will promote 
important outcomes for teaching higher order thinking. 

Implementation 

The importance of the role of the teacher has come up repeatedly in this 
review. Those programs calling for explicit discussion of target abilities and 
skills and that place the teacher in a supportive role for monitoring and devel- 
oping abilities, and skills (e.g, IE, Odyssey, TU. Philosophy for Children, Re- 
ciprocal Teaching, EWP) and developing awareness of control strategies and 
their use (e.g.. Reciprocal Teaching, Philosophy for Children, TU, EWP). seem 
to shov the most promise. Moreover, the better teachers are prepared to use 
the program (e.g., IE, Odyssey, TU, Philosophy for Children, Reciprocal Teach- 
ing) , the better they will be able to manage the complexities involved in 
achieving transfer as well as other important program goals. Teachers must be 
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knowledgeable of specific cognitive objectives and tmderstand how program exer- 
cises provide instances for practice of particular target abilities and 
skills. Ttiey need to know how to foster awareness and use of strategies that 
help students strategically apply their knowledge and skill. Finally, they 
need thorough understanding of subject matter domains to which they will help 
students see the abilities and skills apply. This set of abilities requires 
training beyond what typical elementary classroom teachers have (Table 2: 
Teacher Training) . 

An issue related to teacher training is the extent to which teachers need 
a separate program to teach thinking skills. Since transfer is more likely to 
occur when instruction, practice, and reinforcement occur in a variety of sub- 
ject areas, why not simply teach the skills as they teach the subject areas? 
Program developers argue that having a separate program ensures that time will 
be spent focusing on thinking skills. However, that is only true if the pro- 
gram is implemented as planned (in method and duration), if sufficient time and 
resources are alloved, and if it fits with other priorities in the school. 
They also argue that specific programs help make students more aware that they 
are learning to think and focus more systematically and specifically on develop- 
ing target skills. 

Reciprocal Teaching, EWP, and TU are good examples of infusing rhe teach- 
ing of thinking into regular school work in a systematic way. For instance, in 
Reciprocal Teaching, time and effort are spent focusing on developing key strat- 
egies to improve reading comprehension, while text material is used to practice 
the developing skill. Similarly, the EWP focuses on aocial studies text mate- 
rial while teaching students to better couprehend and write about the mate- 
rial. Such infusion helps students see direct application of the skills to 
helping them learn, instead of having them think that learning to think is an 
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isolated activity. Moreover, using elements of thinking in an integrated way 
provides greater assurance that students' abilities to learn, think, and reason 
are being promoted (Resnick, 1987). 

The guidelines for increasing the likelihood of transfer discussed through- 
out this review (Table 2) point toward the merits of using various programs as 
an aid in teaching thinking, but also infusing the application of such programs 
to subject matter areas (Sternberg, 1987). Developing higher order thinking 
skills and learning when and how to use them is a long-term go ~l that must be 
worked toward throughout the elementary (and secondary) years of schooling. 
The infusion of teaching for specific abilities, skills, knowleage of thinking, 
and appropriate attitudes and dispositions into subject matter domains (through 
using programs and through asking students to think critically abou^ subject 
mar.te.r) is the most likely way to achieve long Cenn effects and to show stu- 
dents how particular skills and abilities apply to different learning experi- 
ences (Alexander & Judy, in press). In both instances, more time devoted in 
the teaching day to such outcomes is required. 

Such infusion of specific programs into regular teaching routines is becom- 
ing more common and widespread (see Brandt 1988) . Many of the newer practices 
in using programs designed to develop thinking skills stem from problems 
identified with isolated skill instruction, and from logistical problems of 
finding time in the school curriculum to add yet another skill area thot takes 
time in the school day. Current trends are moving in the direction of using 
programs like TU to organize thinking skill instruction in a systematic way, 
but to make sure students also have the opportunity to regularly apply the 
skills in meaningful ways to learning of subject matter content and to everyday 
reasoning. 
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Evaluation Issues 

A theme throughout this review has been the lack of specific evidence that 
prograiLS promote desired outcomes or that the effects are long term. One rea- 
son for these problems is thRt evalue.ion efforts and analysis of program mate- 
rials and methods have focused on different evaluation questions, which makes 
it difficult to compare programs. Reviewers of the programs tend to focus on 
analyzing several aspects of the program (e.g., underlying assumptions; program 
materials, methods, target audiences, implementation pattams and problems, and 
teacher qualifications) U. addition to paying attention to program effects 
(e.g., Bransford et al., 1985; Chance, 1986; Nickerson al. , 1985; Poison & 
Jeffries, 1985; Savell, Twohig, & Rachford, 1986; Sternberg & Bhana, 1986). 
They conclude that despite the lack of clear evidence, many of the programs 
show promise, seem to do seme good, and are at least able to provide anecdotal 
evidence that students are learning from the program. Much of this optimism in 
the face of little evidence stems from these reviewers' knowledge and under- 
standing of evaluation problems. 

There are many obstacles to evaluation that program developers or school 
districts must face (Bransford, Bums, Delclos, & 'ye, 1986; Nickerson et al. , 
1985; Poison & Jeffries, 1985; Sternberg & Bhana, 1986). First, sumraative 
evaluation data are difficult to collect. Evaluators must contend with control 
issues (e.g., defining appropriate control group treatments; contrcsli.ing for 
the quality of teaching; controlling for consistency in program implementa- 
tion) . In addition, information regarding generalized and long-term eifects 
requires collecting data in situations that are different from the context in 
which the program was taught anl requires collection points across a subs tan- 
tial amount of time. These requirements are difficult to meet and may be 
impossible for evaluators to control. In f:ddition, a major problem with some 
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program evaluations has been that programs try to assess transfer of skills 
without first determining whether students learned the target skills. 

A second major problem is identifying adequate measures for determining 
whether thinking ability has improved. There is no taxonomy of problem- solving 
and comprehension techniques or of metacognitive skills. Therefore, numerous 
measures have been used to evaluate program effects, and chere are great differ- 
ences in opinion as to which ones are adequate. Since so many different mea- 
sures have been used, cross -program comparisons are difficult or ^possible to 
make. Also, there are problems with whether outcome measures overlap program 
content, so that programs are accused of "teaching to the test," and transfer 
of learning cannot be assessed. Moreover, standard tests may not measure cer- 
tain aspects of desired outcomes. For example, judging the merits of a plau- 
sible argument is an important skill, but difficult to measure with available 
instrximents. An additional complication comes from comparing pre- and posttest 
measures for statistically significant gains. Such gains do not provide 
guidelines for deciding whether the effects are sufficient to justify the cost 
(in time and resources) for implementing the program. Finally, standard 
measures do not address unintended positive effects (e.g., improved attitudes, 
improved quality of classroom discussions) or negative effects (e.g., not fully 
meeting goals is discouraging and unmotivating to teachers and students). 
Other ways must be devised to measure these kinds of effects. 

Hiese are examples of obstacles to evaluations of program effectiveness 
that explain the lack of evidence co support program claims. Kowever, that 
does not mean evaluations should not be done. One kind of evaluation that has 
not yet been discussed is formative or ongoing evaluation of both the program's 
effectiveness and of group and individual progress throughout instruction. 
These are also important sources of information for evaluating success and 
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providing ideas for program improvement. The second Kind, summative evalua- 
tion, must be carried out more extensively and carefully if we are to under- 
stand better the long-term effects of such efforts. Program developers and 
school districts need to make use of available guidelines for conducting 
fruitful evaluations (e.g., Baron, 1987; Nickerson et al., 1985) and more 
carefully document the successes that have occurred. The evaluations that have 
been conducted on Reciprocal Teaching (e.g.. Brown & Palincsar, 1985) and 
Odyssey (e.g., Hermstein, Nickerson, Sanchez, & Swets, 1986) are useful ex- 
amples of rigorous efforts to study several aspects of a program, and provide 
inspiration that such evaluations can and should take place. 

Concluding Remarks 

This review makes apparent the need for multiple goals for teaching higher 
order thinking skills that include the explicit goal of teaching for transfer 
of the skills to a variety of subject area domains. An important ingredient 
for promoting the likelihood of transfer is teaching skills, metacognitive 
awareness and strategies, conditional knowledge and attitudes as they relate to 
various subject matter areas, rather than relying on students to make such con- 
nections themselves (Alexander & Judy, in press). This requires careful ex- 
amination of the extent to which general skills appropriately apply to 
different subject areas, and showing students how, for example, the skill of 
analysis in science- -for example, identify the features of animate and 
inanimate objects--is *;imilar to or different from the skill of analysis of 
literatu^---for example, identify the components of persuasive discourse (see 
Quellmalz, 1987). In addition, specific attention must be paid to helping 
students develop strategic knowledge and ability to manage their own learning 
and thinking. 
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From the available evidence, a clear choice does not emerge as to whether 
teaching thinking skills should take place in specific programs or as part of 
regular subject matter instruction. Instead, an argument has been advanced 
that particular features of programs are necessary if the Ukelihood of trans- 
ferring skills beyond program exercises is to be increased. It was shown that 
many of the programs reviewed can be useful tools, but they need to V in 
conjunction with occasions for students to develop and use high chink- 
ing in subject areas, and that students must be helped to see * .ections 
between the skills and abilities taught in the programs and how they can be 
used in academic learning experiences and everyday reasoning. 
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